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Abstract 

Consider a balls-in-bins process in which each new ball goes into a given bin with 
probability proportional to f(n), where n is the number of balls currently in the bin 
and / is a fixed positive function. It is known that these so-called balls-in-bins processes 
with feedback have a monopolistic regime: if f(x) = x p for p > 1, then there is a finite 
time after which one of the bins will receive all incoming balls. 

Our goal in this paper is to quantify the onset of monopoly. We show that the initial 
number of balls is large and bin 1 starts with a fraction a > 1/2 of the balls, then with 
very high probability its share of the total number of balls never decreases significantly 
below a. Thus a bin that obtains more than half of the balls at a "large time" will most 
likely preserve its position of leadership. However, the probability that the winning bin 
has a non-negligible advantage after n balls are in the system is ~ const, x n}~ p , and 
the number of balls in the losing bin has a power-law tail. Similar results also hold for 
more general functions /. 

1 Introduction 

Consider a discrete-time Markov process with B bins, each one of which containing I%{pi) > 
balls at time m for each m G {0, 1,2,.. .} and i G {1, . . . , B}. Its evolution is as follows: 
at each time m > 0, a ball is added to a bin i m , so that Ii m {m) = Ii m (m — 1) + 1 and 
Ijijn) = Ij(m — 1) for all i G {1, . . . , B}\{i m }, and the random choice of bin i m has 
distribution 

Pr(i m = i\{I j (m-l) : l<j<B})= Wf™'^ (l<i<B), (1) 

£, = i/(Ij(™- 1)) 
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for a fixed positive function / : N — * (0, +00). This recipe specifies what we call a balls-in- 
bins process with feedback function f and B bins, and one should notice that when / is an 
increasing function — the case that has been mostly considered in the literature — there is 
a tendency that the rich get richer, the more balls a bin has, the more likely it is to receive 
the next ball. 

This class of processes^ was proposed by Drinea, Frieze and Mitzenmacher [3] as a model 
for competing products in an economy, as well as a simpler variant of so-called preferential- 
attachment models for large networks (see pQ for a survey of the latter). That paper 
concentrates on the special case where f{x) = x p for some parameter p > 0. The authors 
prove that when p > 1, there almost surely exists one bin that gets all but a negligible 
fraction of the balls in the large-time limit; whereas for p < 1, the asymptotic fractions of 
balls in each bin are all the same. The p = 1 case is the classic Polya Urn model, for which 
it has been long known that the number of balls in each bin converges almost surely to a 
non-degenerate random variable, and thus the process has different regimes depending on 
whether p < 1, p = 1 or p > 1. 

However, stronger results on the p > 1 case are available. A paper by Khanin and 
Khanin [5] introduced what amounts to the same process as a model for neuron growth, 
and proved the following stronger result: if p > 1, there almost surely is some bin that gets 
all but finitely many balls. That is to say, consider the following event, in which bin i is the 
only one to receive balls after some finite time M (we call this monopoly by bin i). 

Moni = {3M G N Vm > M Vj G [B] j / i Ij(m) = Ij(M)} (2) 
= {3M G N Vm > M i m = i}. (3) 

The result of [5] - or rather, a straightforward extension of it proven in |101 [7j - says that 

Theorem 1 (From [5], I10L \7\) If {I^^^q is a balls-in-bins process with B bins and feed- 
back function f = f(x) > satisfying 

+00 ^ 

,Si <+co ' (4) 

satisfies 

Pr (3i G [B] : Mom) = 1. 

In particular, this holds for f(x) = x p , p > 1. 

This is a much stronger statement than the one that Drinea et al. proved in [3], and we take 
it as our starting point in the present paper. We sketch a proof of Theorem [1] in Section \3. 
below, since it helps to build some intuition for our own results. 



A longer background discussion is available from the first author's PhD thesis [7]. 
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Our main goal is to quantify the monopoly phenomenon in the case where there are 
B = 2 bins. As powerful as Theorem Q] is, it tells us nothing about how fast the system 
approaches this asymptotic regime. Informally, we will be interested in questions like: 
suppose we throw in a million balls into B = 2 bins. Should we expect to find a bin with 
10% more balls than the other bin? And in case that does happen, is the leading bin likely 
to lose its lead as we add more and more balls into the system? This paper treats rigorous 
forms of those problems, for a broad (but not entirely general) class of feedback functions. 
A summary of our results is given below. 

1. Imbalanced start. Start the process with a total of t 3> 1 balls in the two bins and 
at least a = 52% of the balls in bin 2. We show in Section [5] that with very high 
probability, there is no future time at which bin 2 will have (3 = 51% of the total 
number of balls in this balls-in-bins process. A similar result holds for any other 
a> (3> 1/2. 

2. Balls in the losing bin. Let L be the number of balls that go into the losing bin, i.e. 
the bin that does not achieve monopoly, when the initial number of balls is fixed. 
We prove in Section [6] that the distribution of L has a heavy tail. More concretely, if 
f(x) = x p for some p > 1, Pr (L > n) ~ c p x nr~ v for large n. 

3. The time until imbalance. We show in Section [7] that for fixed initial conditions, the 
probability that the losing bin has at least cm balls at time n (for a < 1/2 fixed) also 
decays slowly in n. In particular, in the case f(x) = x p , this probability is ~ c' p x n 1_p 
for n large; that is, we have a power law with the same exponent as in 2.. An extension 
of this result is presented in Section [H 

The picture that emerges from these results is that it takes a long time before a clear 
leader emerges, but once it does, it is likely to stick. One indication of the heavy-tail part of 
our results was in the numerical simulations of [3], which indicated that the a clear leader 
of the process took a long time to emerge. Reference [I] also showed that once the leader 
emerges, it achieves dominance very quickly, but our results regrading this (cf. 1.) are 
stronger. 

Our main technical tool has been employed in [10] and other works, and seems to have 
originated in Davis' work on reinforced random walks [3]. We shall embed the discrete-time 
process we are interested in into a continuous-time process built from exponentially dis- 
tributed random variables. The most salient feature of this so-called exponential embedding 
is that arrival times at different bins are independent and have an explicit distribution. 
This greatly simplifies calculations and permits the use of Chernoff-like bounds that we 
develop below. 

The remainder of the paper is organized as follows. We discuss preliminary material in 
Section [21 Section [3] rigorously introduces the exponential embedding process and discusses 
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its key properties (including Theorem [T]) . In Section 0] we detail the assumptions we make 
on our feedback functions /, while also deriving some consequences of those assumptions. 
The next three sections correspond to items 1. — 3. . Section [8] discusses possible extensions 
to our results and some related work. 

Acknowledgements. I thank Michael Mitzenmacher and Eleni Drinea at Harvard 
University for early discussions regarding this work. Thanks also go to my former Ph.D. 
advisor Joel Spencer, who introduced me to this topic and made numerous valuable sug- 
gestions to the work presented here. 

2 Preliminaries 

Set notation. Throughout the paper, N = {1,2,3,...} is the set of positive integers, 
R + = [0, +oo) is the set of non-negative reals, and for any k G N [A;] = {1, . . . , k}. 

Asymptotics. We will use the standard O (•) jo (•) /U (•) / <C / ~ asymptotic notation. 
Let f,g be functions of a real parameter t and to be a limit point of the domain of the 
two functions. We will say that f{t) = O (g(t)) (or equivalently g(t) = f2 (/(t))) as t — > to 
when limsup t ^ to \ f(t)/g(t)\ < +oo. We will also say that f(t) = o(g(t)) (or f(t) g(t)) 
as t — > to when limsup^^ \f(t)/g(t)\ = 0. Finally, we will say that f(t) ~ g(t) as t — > to 
when lim^ f{t)/g(t) = 1. 

Balls-in-bins. Formally, a balls-in-bins process with feedback function / : N — > (0 + oo) 
and BgN bins is a discrete-time Markov chain {(Ji(m), . . . , /£(m))}m=o with state space 
N B and transition probabilities given in the Introduction (see (pQ)). We will usually refer 
to the index i m £ [B] as the bin that receives a ball at time m. 

If B = 2, n G N and < a < 1, we sometimes denote by [n, a] the state ([cm],n — 
[an]) G N 2 , i.e. there are n bins in the system and an a- fraction of them is in bin 1 (with 
rounding). This alternative notation will be used whenever convenient. 

For any B, if E is an event of the process and u G N s , Pr u (E) is the probability of E 
when the initial conditions are set to u. The same notation will be used for the exponential 
embedding defined in Section [3j 

Exponential random variables. X = d exp(A) means that X is a random variable with 
exponential distribution with rate A > 0, meaning that X > and 

Pr(X >t) = e~ xt (t > 0). 

The shorthand exp(A) will also denote a generic random variable with that distribution. 
Some elementary but extremely useful properties of those random variables include 
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1. Lack of memory. Let X = d exp(A) and Z > be independent from X. The distribu- 
tion of X — Z conditioned on X > Z is still equal to exp(A). 

2. Minimum property. Let {X{ = d exp(Aj)}™ =1 be independent. Then 

Y - — ™- Y — d 



, mjll = min Xi= exp(Ai + A 2 + . . . K. 

Ki<m 



and for all 1 < i < m 



Pr(Xi = X n 



(5) 



Ai + A2 + • • • A m 

3. Multiplication property. If X = d exp(A) and 77 > is a fixed number, 77X = d exp(A/r/) 

4. Moments and transforms. If X = d exp(A), r G N and t£R, 



Ex [X r 



Ex 



Ex 



JX 



A r 



1 - 



-it 



T^t (t<\) 
+00 (t > A) 



(6) 
(7) 

(8) 



3 The exponential embedding 
3.1 Definition and key properties 

Let / : N — > (0, +00) be a function, B G N and (ai, . . . , as) G N 5 . We define below a 
continuous-time process with state space (N U {+oo}) B and initial state (ai,...,as) as 
follows. Consider a set {X(i,j) : i £ [B], j £ N} of independent random variables, with 
X(i,j) = d exp(/(j)) for all € [B] x N, and define 



n— 1 



Ni(t) = sup < ra G N : V < t> (i G [B], t G R + = [0, +00)), 



j=a, 



(9) 



where by definition Ylj=i(- ■ ■) = if i > k. Thus iVj(O) = Oj for each i G [£], and one 
could well have iVj(T) = +00 for some finite time T (indeed, that u>iZ/ happen for our 
cases of interest); but in any case, the above defines a continuous-time stochastic process, 
and in fact the {iVj(-)}^ =1 processes are independent. Each one of this processes is said to 
correspond to bin i, and each one of the times 



X(i, Oi),X(i, ai ) + X(i, ai + 1), X(i, ai ) + X(i, a { + 1) + X(i, m + 2), . . . 
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is said to be an arrival time at bin i. As in the balls-in-bins process, we imagine that each 
arrival correspond to a ball being placed in bin i. 

In fact, we claim that this process is related as follows to the balls-in-bins process with 
feedback function /, B bins and initial conditions (oj, . . . , «b). 

Theorem 2 (Proven in [3], [S], IIP), ]7\ [§]) Let the {-/Vj(-)}ie[B] process be defined as above. 
One can order the arrival times of the B bins in increasing order ( up to their first accu- 
mulation point, if they do accumulate) so that T\ < T2 < ■ ■ ■ is the resulting sequence. The 
distribution of 

{I m = (N 1 (T m ),N 2 (T m ), . . . ,N B (T m ))} m&N 

is the same as that of a balls-in-bins process with feedback function f and initial conditions 
(ai,a 2 , . . . ,a B ). 

One can prove this resultd as follows. First, notice that the first arrival time T\ is 
the minimum of X(j,aj), (1 < j < B). By the minimum property presented above, 
the probability that bin i is the one at which the arrival happens is like the first arrival 
probability in the corresponding balls-in-bins process with feedback: 

Pr (x(i, ai ) = min X(j,aj) \ = . (10) 

More generally, let t G R + and condition on (Ni(t))f =l = {bi)f =l G N 5 , with b{ > at for 
each i (in which case the process has not blown up). This amounts to conditioning on 

6j — 1 bi 
Vi G [B] bi)<t<J2 X (h bi). 

j=di j = CLi 

From the lack of memory property of exponentials, one can deduce that the first arrival 
after time t at a given bin i will happen at a exp(/(6j))-distributed time, independently for 
different bins. This takes us back to the situation of (|10p . only with bi replacing a,, and we 
can similarly deduce that bin i gets the next ball with the desired probability, 

f(h) 

2 The exact attribution of this result is somewhat confusing. Ref. [5] cites the work of Davis [3] on 
reinforced random walks, where it is in turn attributed to Rubin. 
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3.2 The occurrence of monopoly 

The exponential embedding yields a "Book proof of Theorem[TJ Without loss of generality, 
we can assume that B = 2 (the general case follows from comparing pairs of bins). The 
notation we use comes from the previous section, and we also employ the version of the 
balls-in-bins process given by I m = (Ni(T m ))i = i^ (cf. Theorem [2]). 
Under the condition that 

+oo j 

one has 

+oo 

Fi = < +oo almost surely (i = 1,2). (12) 

Indeed, the terms in Fi are all positive and since X(i,j) = d exp(/(j')) for all i,j, 

+oo 

Ex [^] = E77jy<+ 00 b y CID- 

J =a i 

It is also easy to see that the F^s are independent and have no point masses in their 
distributions. Thus with probability 1, either F\ < F2 or F 2 < F\. 

Suppose that the first alternative holds. Since Ylj^ f Fi as N — ► +00, we can 

deduce that 

n— 1 ri2 — 1 

3n 2 £ N Vn > ai A ljU = ^ X(l,j) < F 1 < A 2 ^ 2 = £ X(2,j). (13) 

i=aa ^=a2 

The sequence {j4i, n }neN is composed of arrival times; that is to say, it is a subsequence of 
{TmjmeN- Moreover, that sequence converges to F\. It follows that the first accumulation 
point of the sequence {T m } mg N is at most F\, and that T m < F\ for all m. But since 
A.2,n 2 > Fit this implies that for all times m of the discrete-time process, 

I m {2) = N 2 {T m ) < n 2 . 

Thus if F\ < F 2 , there exists a finite n-2 G N such that I m (2) < n 2 for all m, i.e. bin 2 never 
has more than n 2 balls. This implies that bin 1 must achieve monopoly whenever F\ < F 2 . 

If F% > F 2 , the same reasoning shows that bin 2 achieves monopoly. As pointed out 
above, with probability 1 either F\ < F 2 or F 2 > F%; thus the proof is finished. 

Remark 1 It is not hard to show that Fi = +00 almost surely if2~2j f{j)~ l = an d i- n 
that case monopoly has probability 0. The interested reader can see J21 [7| \B$ for details. 
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Remark 2 Assume that bin 1 achieves monopoly, as in the proof above. In this case, all 
arrivals of the continuous-time process at bin 2 after time F% do not actually happen in 
the embedded discrete-time process {I m = (I m (l), J m (2))}. We call these "ghost events" a 
fictitious continuation of our process. This very useful device is akin to the continuation of 
a Galton- Watson process beyond its extinction time (see e.g. [2]) and is equally useful in 
calculations and proofs. 



4 Assumptions on feedback functions and a large-deviations 
bound 

The purpose of this rather technical section is two-fold. First, we spell out the technical 
assumptions on the feedback function / that we need in our proofs. Nothing seems to 
actually require these assumptions, but they facilitate certain estimates that we employ in 
the proofs. 

The second purpose is to present a large-deviations bound on random sums such as 
Y^j^nX(i,j). One can check that the variance of such a sum decreases as m — > +oo; 
we show below that these sums are in fact close to their means with all-but-exponential 
probability. 

Some readers might wish to skip the proofs in this section on a first reading. 



4.1 Valid feedback functions 

The feedback functions we allow in our results satisfy the following definition. 

Definition 1 An increasing function f : N — > (0, +oo) with /(l) = id is said to be a valid 
feedback function if it can be extended to a C 1 function g : [l,+oo) — > (0,+oo) with the 
following property: if (lng(-))' is the right- derivative of In g, and h{x) = ^(ln^x))' (for 
x £ R + U {0} 

1. liminf x .^ +00 h(x) > I; 

2. lim^+oo x-V 4 h(x) = 0; 

3. there exist C > and xq £ R + such that for all e G (0, 1) and all x > xq 

h(t) 



sup 

x<t<x 1 + e 



h(x) 



1 



< Ce. 



(14) 



With slight abuse of notation, we will always assume that f is defined over [l,+oo) and is 
C . We will also call h the characteristic exponent of f. 



3 The requirement that /(l) = 1 is just a normalization condition, as it does not change the process. 
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Functions with exponential growth (such as f(x) = 2 X ) or with oscillations fail to satisfy 
Definition [TJ On the other hand, requiring that / be increasing seems natural, and the 
assumption still leaves us with plenty of interesting examples of feedback functions. For 
instance, any of the functions defined below 

f(x) = x p ( for some fixed p > 1), 

f(x) = x pln x ( for some fixed p > 1, a > 0), 

f{x) = x p Xn.(x + e — 1) ( for some fixed p > 1). 

is valid. The "canonical case" where fix) = x p (x > 1) explains the terminology for the 
characteristic exponent: in that case, h(x) = p for all x > 1. We also note that whenever 
/ is a valid feedback function, the monopoly condition is satisfied. 

Proposition 1 If f is a valid feedback function, X^'eN /CO 1 < +oo. 

Proof: The condition limmf x ^ +OQ h(x) > 1 implies that there exists a n £ N such that 
h(x) > c > 1 for all x > n. This implies that f(j) = Q, (j c ) as j — > +oo, which is enough 
for the convergence of J2j£N fti)- n 



4.2 Consequences of the definition 

Let us now define the quantity 

m— 1 

SV(n,m)=V— — (r G R + , n G N,m G NU {+oo}) (15) 
j(n) r 

j=n J v ' 

for some / : N — » (0, +oo), and also let S r (n) = S r (n, +oo) (which might diverge for some 
r). If f(x) = x p and r > 1, a simple calculation shows that for n S> 1 



5 r (n) 



+°° dx __ n x ~ Tp - m l ~ rp 
n f(x) r (rp - 1) 



The main content of the following lemma is that a similar result holds for any valid /, if p 
is replaced by the characteristic exponent h. 

Lemma 1 Assume that f is a valid feedback function with characteristic exponent h. De- 
fine the possibly divergent integrals: 

r+oo (] r 

M r (n)= — — (r G R + , n G N). 

Jn J\ x ) 
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Then for all r > 1 both S r (n) and M r {n) converge and moreover, as n —* +00 



SV(ra) ~ M r (n) 



(rh(n) - l)/(n) 
anrf /or all fixed r > 1, p > 1 there exists a < 1 suc/i i/iai 

M r (px) < aM r (x) for all large x 
Thus for any r > 1 anrf p > 1 : 

n 



S r (n, \pn\ ) = VL 



(rh(n) - l)/(n) 



Before we present the proof of Lemma dj we state two other lemmas. They follow 
directly from the assumptions on h(x) and we omit their proofs. 

1 /4 

Lemma 2 Si (re) 3> e~ n as re — > +00. 

Lemma 3 For any bounded function w : N — > R ; /(re + w(n)) ~ /(re) anrf /i(re + w(n)) ~ 
/i(n) as n — > +cxd. 

Proof: [of Lemma U] Essentially the same proof appears in the last result in [6], but we 
reproduce the argument here for convenience. Under our assumptions, F(-) = f(-) r is a 
valid feedback function with characteristic exponent rh(-). Clearly, the lemma holds for / 
iff it holds for F with r = 1. It follows that it suffices to prove the lemma in the case r = 1, 
which is what we do below. 

We can assume that re is large enough, so that inf x > n h(x) = c > 1, and moreover 



Vx > n,Ve > sup 

x<t<x x + t 



hit) i 



h(x 



< Ce, (16) 



which follows from the assumptions on h in Definition [TJ We start by noting that for all 

y > x > n 



c 



f(y)>(y\ (17) 



m 

To see this, it suffices to notice that 
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Inequality (fT7|) implies that f(x) S> x as x — ► +00. This justifies the following integration 
by parts procedure. 

+ 7„ exp(ln/(x)) { ' 



x— >+oo 



n / ,+0 ° /i(x) . , 

+ / J \ ( 2 °) 



/(») Jn f(x 

where in the last line we plugged in the definition of h. We now claim that 
f+°° h(x) dx , , , r+°° dx , , N , , , , 

/ -^r /i(n) / — - = h{n) M x (n) as n -c +00. (21) 

We will prove (|2ip below, but first we show how it implies the lemma. Employing (|2ip with 
equations (fT8l) to ([20]) shows that 

Ml ( n ) ~ ^7 S TT77 S as n ^ +00, (22) 
(/i(n) - l)/(n) 

since /i(n) > c > 1 for n large (as discussed above). By the smoothness assumption, 
h(n)/n — ► as n — ► +00, and therefore 

n 1 

^ as n — > +00. (23) 



(fc(n) - l)/(n) /(n) 
Now notice that, since / is increasing 

1 



< Afi(n)-5i(n) < 0, 



/(«: 

hence \Mi(n) — Si(n)\ < /(ti)^ 1 and by equations (|22p and 

5i(n) ~ Mi(n) ~ — — 77 s as n ^ +00, 

(/i(n) - l)/(n) 

as desired. Moreover, if p > 1 is fixed, equations ([16]) and ([T7]) imply that, for large n, 
h(\pn\) ~ /i(n) and /([pn]) > p c f{n) with c as above, 

hence 

<(l + o(l»- " 



(ft(M)-l)/(M) " " (/!(n)-l)/(n)- 

from which the desired statement about M r (\pn\) follows. 
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We now prove (|2ip . Choosing an arbitrary (but fixed) e > 0, we first show that 

f+°° h(x) dx f +0 ° h(x) dx , , 

as n -> +00, / « / ^ttV- 24 

Jn^ f{x) Jn f(X) 



Indeed, 



+co h{x)dx f+°° h(u 1+e ) u e du 



l+e 



Jn 



f(x) V 'Jn 



< (1 + C )(1 + C€)/ 

j n 

< (l + e)(l + Ce)n( 1 - c ) e / 

J n 



f(u^) 
+ :X) h(u) u e du 

/(«)«" 
+°° /t(u) 



/(«) 



+00 



/i(u) du 

~7W 



where the first line is a change of variables, the second line employs (|16|) . the third line 
uses (|17p applied to x = u and y = u 1+e , and the remaining lines follow from c > 1. The 
sequence of equations proves (|24|) . which implies in particular that, for n large, 



+°° h(x) dx f nl+c h(x) dx 



f{x) Jn f(x) 
But another use of ([TBI) implies that 



m 1+e h(x) dx 
Jn 

rn 

Jn /(i) 



(1 - Ce)h(n) < "' < (1 + Ce)h(n), 



and, similarly to (|24|) . one can show that 



r "+ 00 dx /•+°° dx 
as n — ► +00, / — - <C 



f{x) Jn f{x 



Thus we conclude that, as n — > +00, 



r+oo h(:r) cfcr 

Jn -j^y- 



1 - Ce - o (1) < i^j < 1 + Ce + o (1) 

Since e > is arbitrary, (|21|) follows, and the proof is finished. □ 
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4.3 A large-deviations estimate 

If bin 1 starts with a± balls, the time until it has b\ > a\ balls in the continuous-time 
process is 

61-1 

E 

j=ai 

and the time until bin 1 acquires infinitely many balls is 

+00 

E 

j=ai 

The latter is a sum of independent exp(/(j)) random variables, and it converges whenever 
J2j f(j) 1 < +°° Section f3.2|) . We will show that the sum concentrates very strongly 
around its mean. 

Lemma 4 Let f be a valid feedback function, so that the monopoly condition J2j f(J) < ~ 
+00 holds (cf. Proposition^. Assume that {Vj = d exp(/(j))} Jg N be a sequence of inde- 
pendent random variables, and define (for n G Nj 



+00 , , , 



Then there exists a constant C = Cf > such that for all large enough n E N and all 
t G R + 



Pr [A n > U S 2 (n)) < Ce 



Pr [A n < -t^S 2 (n) J < Ce- 1 

What Lemma|4]means to us is that Sj^ti ( 1 ? J ) can be though of as "almost constant" in 
many calculations. This will be put to use in all of our main proofs. In fact, the following 
corollary will suffice. 

Corollary 1 Let f and {Vj}j^ be as above, and define 

+00 

j=n 

Then Ex [B n ] = S± (n) and there exists a C = Cf > a such that 

a 



Vn 6 N, Pr 



Si(n) 



n i 



< C'e~ nl 
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Proof: [of Lemma 0] We will only prove the first inequality, since the other proof is simi- 
lar. The technique we employ is fairly standard and is commonly used in other proofs of 
Chernoff-type large deviation inequalities [2]. 

Fix any < s < /(n)/2 = minj> n f(j)/2 and notice that, by the standard Bernstein's 
trick, the formulae in Sectional the inequality "1 + x < e x " , and some simple calculations 



Pr[A n > tJS 2 {n) 



— p r ye sA ™ > e st \/*(n) 



e J2j>„ s ( V i f(j)) 



3>n 



j>n 1 ~ 7U) 
e -sty/ 5 2 (n) x 

T T « I S S 

n e m 1 + + 



< 



e-^v^W JJexp(2 



/(i) 2 



exp(2s 5 2 (n) - stJ ^(n)) 



We now set 



If we show that this choice is permissible (i.e. that S 2 (n) < /(n)/2 for n large), we 
will have finished the proof. But notice that 

(2h(n) - l)/(n) 2 



1 



« f(n). 



5 2 (n) n 

since h(n) <C n 1 / 4 by assumption. We deduce that there is some no = no(f) such that for 
all n> riQ l/S^n) < /(n) 2 /4, from which the lemma follows. □ 

Proof: [of Corollary [1] Recall that in this case < B n < +oo, since B n is positive and has 
finite expectation. Hence, if A n is as in Lemma HI B n = A n + S\(n). The Corollary follows 
from Lemma U] by the choice of t = n 1 / 4 and recalling Lemma [H which implies that 



5 2 (n) 1 / 2 = O f 5i(n)^//i(n)/n) = O (rT^S^nfj as fc(n) < n 1 / 4 by assumpt 



ion. 



□ 
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5 Imbalanced start 

We now start the discussion of the first of our main results. Recall the notation [n, a] for 
the state of a two-bin balls- in-bins process with feedback (cf. Section [2]). Our interest in 
this section will be in processes with two bins, started from [n, a] with n large, a G [0, 1/2) 
fixed, and / valid. To state this theorem, we need a definition. 

Definition 2 Let (3 £ (0,1/2) and N 6 N be given, and consider a balls-in-bins process 
with two bins. HasMoreThan(/3, N) is the event that there are more than (5N balls in bin 1 
at the moment when there is a total of N balls in both bins. 

Theorem 3 Suppose that f is a valid feedback function, so that in particular X^jeN f(J) < 
+oo and monopoly has probability 1 (cf. Proposition [/J). Then for all < a < (3 < 1/2, 
there exists a constant 7 > depending only on a, [3 and f such that for all large enough 



In particular, if the initial conditions are [n, a] as above, bin 2 achieves monopoly with 
all-but- exponentially- small probability. 

Proof: [of Theorem [3] Recall the definition of the exponential embedding: for i = 1,2, 
X(i, 0) parameterizes the time until the first arrival at bin i, while X(i,j) (for j > 0) 
parameterizes the time between the (j — l)th and jth arrivals at bin i. We begin by 
showing the following fact, which will also be useful in Section [71 

Claim 1 Consider a balls-in-bins process with feedback function f and initial conditions 
(x, y) with x + y = n. Let (3 £ (0, 1) and N > n be given. Then 



Proof: [of Claim [T] Let A be the event in the RHS of (|25p . We begin by assuming that 
HasMoreThan(/3, N) occurs and show that this implies the occurrence of A. Consider the 
time tn when the total number of balls in the continuous time process reaches N . At that 
time, the number of balls in bin 1 (respectively 2) is larger than \/3N~\ (resp. smaller than 
iV — \(3N~\), by assumption, so 



Pr [n>Q! ] (3N > n, HasMoreThan(/3, n)) < e 




(25) 



N-\/3N~\-l 



X(l,j)<T N < 



j=x 



l=y 
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which implies the occurrence of A. Conversely, assume that A occurs, and let tn be as 
above. Then, because 

\0N~\-1 N-\0N~\-1 

J2 x(i,j)< y, 

j=x £=y 

the number of balls in bin 2 at time X^f^ 1 ( 1 , J ) is smaller than N — \(3N~\ , so that 

r/3An-i 

]T x(ij)<r N . 

j=x 

This implies that at time tn , the number of balls at bin 1 is at least \(3N] , which implies 
the occurrence of HasMoreThan(/3, N). □ 

We now continue the proof of Theorem[3l Given a number D > independent of n, consider 
the event E n where the four conditions given below hold simultaneously: 



ViV > n, 



Si (\an\) 



1 



si(rw) 



ViV > n, 



5i(n — [an] ) 



5i(jv- rw) 



< Dn 4, 



< Z>]\r*, 



< Dn 



< DN~*, 



(26) 
(27) 
(28) 
(29) 



The above conditions correspond to the class of events covered by Corollary[TJ For instance, 
to get the first condition we may look at the concentration of B^ an ^ = Z)^r an i It 
follows that there exists a D > depending only on C = as in the Corollary, a and j3 
for which: 



Pr (E n ) > 1 - 2C'e 



,1/4 



E 2C>t 



-N 1 / 4 



> 1 - e 



N>n 



with 7 > depending only on /, a, (3. 

Prom now on, our goal will be to show that: 

For all large enough n G N and all N > n, E n n HasMoreThan(/3, N) = 0. (30) 

Notice that this implies the Theorem, as 

Pr (3N > n, HasMoreThan(/3, N)) < Pr (££) < e"™ 7 for all large n. 
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To establish (|30p we note that there is nothing to prove if N — n < \/3N] — \an\ : in 
that case, there cannot be > 1/3N] balls in bin 1 after N — n balls are added to the system. 
Hence we can assume that 

N - \(3N] >n - \an] . (31) 
Inside E n , we can use ([26]) - ([29]) . Lemma Hand Lemma [3] (to get rid of ceilings) to deduce: 

\PN~\-1 +oo +oo 

3= \ an ~\ 3= \<xn~\ 3= \PM~l 

> (l + o(l))Mi(on)-(l + o(l))MiC9JV) 

/-, / 1 w f +oc dx / 1 w f + °° dx 

= (1 + 0(1))/ ----(l + o(l))/ 7rT , 

Jan J[X) Jf3N J [X) 

where the o (1) terms converge to as n — ► +oo, uniformly over N satisfying (|31)) . Similarly, 
we have 

N ~ m] -H» dx . /•+« & 



Y, X(2,j)<(l + o(l)) _(i + (i))/ 



J=n _ [anl -(l-a)n/(x) J(l-PN)f(xy 

Moreover, under initial conditions [n, a] one has 

HasMoreThan(/3,iV) = I ^ X(l,j) < ^ X(2,£) I . (32) 

^j=[on] i=n—\om~\ ) 

It follows that E n n HasMoreThan(/3, N) / implies 

(l+o 1)) / 773-(l+°(l) / T77\ < (l+o 1)) / ^-(l+o 1)) 



(Hi 



/(a:) •/ PN f(x) J(l-a)nf(x) ' J(1-P)N f(x)' 

This is equivalent to: 

£? n nHasMoreThan(/3,JV)#0=^ / < (1 + o (1)) / (33) 

Jan J[X) Jf3N J (X) 

since by Lemma [U there is some a < 1 such that 

/•+00 r+oc d x 

M l (an)= ——< aMx((l - a)n) = a -— 

Jan J{X) J(l-a)n j[x) 

and 

r+oo f] T r+co f\ T 

M 1 (f3N)= < oMi((l-^)n) = a 



PN f(x) " J(1-P)N f(x 
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We will finish the proof by showing that the RHS of (|33|) cannot hold for large n and 
-/V satisfying (|3ip . To this end, we employ estimate (|17p from the proof of Lemma HJ which 
states that 

Vy > x ^Jf \ > f— ^ , where c = inf /i(x') (34) 

j (X ) \^X J x ^ x 

We will only employ this inequality for large x < y, which means we can assume c > 1, 
because / is a valid feedback function. For any iV satisfying (|3ip . 

i 1 -^ dx fn\ c f^-P)N dx 

< 



f(x) ~ \NJ V /(#*) 
n \c-l r(l-0)n dy 



NJ J(3n f(y) 

l-py- 1 f(i-P) n dy 



Pn f (y) ' 

Here, we employed (|34|) for the second line, the substitution x \— * Ny/n in the third line, 
and (|3ip on the third, with (once again) a o (1) term that is uniform over n. Since c > 1 
and (3 > a (hence 1 — /? < 1 — a), there exists a constant < d < 1 such that for all large 
enough n and all N satisfying (f3T|) . 



(1-/3)N dx ,(l-/3)n dy 

< a 



W f( x ) hn f(y) 

To conclude, note that [a, 1 — a] D [(3,1 — P], hence the above implies 

(1-P)N dx r(l-a)n dx 

< d - , . with d < 1 constant, 



W f( x ) Jan f{x) 

which is incompatible with the RHS of (|33p . This finishes the proof. □ 



6 The number of balls in the losing bin 

We now prove the first of our two heavy-tail results. We first recall the definition of L. 

Definition 3 Let f be a feedback function satisfying the monopoly condition J2j f(j) < 
+oo, so that in the corresponding balls-in-bins process there almost surely is one bin that 
receives all but finitely many balls. For a two-bin process with feedback function f , the losing 
number L is the (almost surely finite) number of balls that go into the remaining bin. 

Our heavy tails result for L is stated and proved below. 
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Theorem 4 Let f be a valid feedback function, in which case the monopoly condition 
12j /(i) 1 < +°° *s satisfied (cf. Proposition^). For any fixed initial conditions (x,y), 
there exists a number c > (depending only on x, y and f) such that, as n — > +oo, 

n 

Pr (9a) (L > n) ~ c5l (n) ~c (fe(n) _ 1)/(n) > 
where h(n) is the characteristic exponent of f (cf. Definition^ and LemmaU^. 

In the case /(n) = p > 1, S'i(n) ~ n 1 ~ p /(p — 1) for n large, and as a consequence we 
have the following corollary. 

Corollary 2 For /(re) ~ ra p with p > 1 and (x, y) 6 N 2 /ixed, as n — > +oo 

Pr (aw) (L > n) ~ (p _ 1 C )nP _ 1 , 

mf/i c as above. 

Proof: [of Theorem H] We will assume without loss of generality that x > y. First, notice 
that for any n > x 

{n oo oo 

X>(V)<X>(i,i)<I>(V) 
l=y j=x £=y 

{n oo oo 

£x(U)<£x(V)<X>(U) 

Indeed, L > n if and only if bin 1 explodes first, but bin 2 has at least n balls when that 
happens, or vice-versa. Define 



n— 1 n— 1 



oo 



4 1 ) = ^x(i,i) 

j=ra 

oo 

4 2) = E*( 2 ^) 



These random variables are independent, and one can rewrite 

{L > n} = {0 < A n < B®} U {0 > A n > -B^}. 
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Bn^ and Bn have the same distribution and are independent of A n , and the distribution 
of A n has no point masses. It follows that 



Pr (a . j,) (L > n) = Pr (o < A n < B^) + Pr (a , x) (o > A n > -£« 
= Pr(0< |A n | <B n ) 
= Pr(|AJ<B n ) 



where we let B n = B^ for simplicity. 

We now apply the concentration result, Corollary [[J to B n = 2~Zj=^ Vj with Vj = X(l,j 
It follows that there exists some C > depending only on / such that: 



Pr 



Br, 



Si(n) 



1 



> 



n 



< C'e 



lV4 



Plugging this into the previous equation yields 

Pr (|A„| < (1 - C'n^ 4 )Si(n)^ - C'e 



7. 1 / 4 



< Fr (x, y ) (L> n) < 
Pr (|A n | < (1 + C'rrV^S^nj) + C'e~ nl/ \ 



(35) 



We will eventually show that there exist a sequence {c n } ng N and a constant C > 0, 
both of which depend only on /, x and y, such that 

|Pr(|A„,| <e)-c n e\ < Ce 3 , (36) 

and c n — * c for some real- valued c > 0. This inequality implies that, as n — > +00, 

|Pr(|A n | <{l±C'n~ 1 ' A )S 1 {n)) -c^n)] = O (rT^S^n) + S^nf 

Since we know that e~ nl/A <C S\{n) <C 1, this implies the Theorem via f)35[) . 



To prove 



first let VVi(') be the characteristic function of A n . 
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Clearly, for all n £ N A n has a distribution with no point masses, hence the inversion 
formula for characteristic functions (a.k.a. Fourier inversion formula) implies 

V-oo<a<6< +00 Pr (a < A n < b) 



— lim 



2-7T T/+00 J-T 



-lta g— yj — ltb 

sf-Lt 



dt. 



In the present setting, we use this formula with b = — a = e, to prove (f36]) . noting that, 
since ip n is integrable, we can dispense with the limit in T. 



1 



e x Pr (|AJ < e) = - lim 



-let 



-let 



IT T/+00 J-T 

1 f + °° , . , sin(et) , 
7r J-00 et 



dt 



Now notice that, quite crudely, 



V,GR,|^1-1|<C S 2 

s 

for some constant C > 0. Applying this inequality with s = et, one obtains 

r+00 r +oo sin(et) 



/+00 r 
if> n (t) dt < / 
-OO J — 



\Mt)\ 



et 



dt 



- Ce< 



+00 



\t 2 Mt)\dt. 



For n = x + 3, \t 2 tp n (t)\ is of order 1/t 2 , so the above integral converges; for n > x + 3, the 
integrand is even smaller. Hence, we can guarantee that, for a possibly larger C > 0, 



PrflAJ < e) 



I f+co 



vr J- 



Mt)dt)e 



<Ce uniformly over n. 



Moreover, since | , n+ i(t)| < |^ n (f)| for all t, the Dominated Convergence Theorem implies 
that 



I r+oo 



7T 



lp n (t) dt 



\ r+oo 
C=~ 

vr J-00 



'cc-l 

n 



1 



1 _i_ v^t 



'n-l 

n 



1 



1 * 2 



dt. 



(37) 



Our last step is to prove that c = lim n c n > 0. To see this, consider first the case x = y. 
In this case, the formula for c in ([371) becomes 



+00 / +°° 

n 



1 



1 * 2 
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The product in the integrand converges to a positive limit for all i 6 R, since fU) 1 < 
+00; hence, the value of the integral is positive, and we are done in this case. 

We now consider the case y < x. Clearly, c = lim n ^ +00 Si(n) _1 Pr (L > n) is a real 
number. Moreover, notice that there is a positive probability a that in the process started 
from (x,y), bin 2 receives the first x — y balls, thus evolving to state (x,x). Conditioned 
on that happening, the probability of L > n is Pr^ ^ (L > n) ~ d S\{n) for some d > 
(as shown above). But then 

„ - i im Pr W)( L>n ) 



n-*+oo Sx(n) 
> Pr( x y ) (bin 2 gets first x — y balls) 

_ Pr^y) (L > n I bin 2 gets first x — y balls) 

»w+oo 5*1 (n) 

= a iim -. , = ac > L) 

which proves that c is positive even when x and y differ, thus finishing the proof. □ 



7 The time until imbalance 

The strategy used to prove Theorem|4]was very simple. After the event {L > n} was written 
down in terms of the exponential embedding random variables, B n was approximated by its 
expectation, and the distribution of A n near the origin via Fourier transform techniques. 
As we shall see below, our last theorem in this section has a similar proof. First, we need 
a definition. 

Definition 4 Given a number n S N, a number < a < 1/2 and a balls-in-bins process 
with two bins, the event LoserHasMoreThan(a, n) holds if at the time the number of balls in 
the system reaches n, the number of balls in the losing bin is at least an. 

Our second heavy-tails result can now be properly stated. 

Theorem 5 Let f be a valid feedback function, in which case the monopoly condition 
J2j f(j)^ 1 < +°° i s satisfied (cf. Proposition^. Then for all fixed initial conditions 
(x,y) 6 N 2 and < a < 1/2 there is a constant c > depending only on f, x and y such 
that 

Pr/ X ^ (LoserHasMoreThan(a, n)) ~ c[Si(|~an] , n — [an])] as n — > +00. 

Moreover, we can take c to be the same constant (depending on f, x and y) that appears in 
the proof of Theorem [7} 
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In the case f(x) = x p for p > 1, the estimate of Si (re) by an integral implies the following 
corollary. 



Corollary 3 For f(n) = n p with p > 1 and (x,y) £ N 2 , < a < 1/2 fixed, as n — > +oo 



with c as above. 

Proof: [of Theorem [5] We follow the same outline as in the proof of Theorem HI We will 
again assume that x > y, and write the event under consideration in terms of the ran- 
dom variables in the definition of the exponential embedding. To do that, first notice that 
LoserHasMoreThan(a, n) occurs if and only if both bins have at least an balls at the time 
when the total number of balls in the system is n. Indeed, if one of the bins has less than 
an < n/2 balls, then this must necessarily be the losing bin, and LoserHasMoreThan(a, n) 
cannot occur in this case. Conversely, if both bins have at least an balls, then in particular 
the losing bin has > an balls, and LoserHasMoreThan(a, n) occurs. 

The event that bin 1 has at least an balls when n balls are present in the system is 
precisely the event HasMoreThan(a, n) defined in Definition [21 which was shown in Claim [1] 
to be equal to 



Similarly, the event than bin 2 has at least an balls when there are n balls in the system 
is precisely HasMoreThan(a, n) with the roles of the two bins reversed, and can thus be 
written down as 



Pr( Xjy \ (LoserHasMoreThan(a, n)) 




- p - (1 - af-P) 



(p - 1) nP- 1 





We conclude that 



LoserHasMoreThan(a, n) 
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[an] — 1 [an] — 1 n— [an] — 1 

E x(i,j)- E x(2,£)< ^ x ( 2 ^) 

3=9! j=y l=\an\ 

{[an] — 1 [an] — 1 n— [an] — 1 

E X(2,Z)- E ^.3)< E 
^=3/ 3=2: j=\an\ 



(38) 



where for the last equality we used the fact that a < 1/2, which guarantees \an\ < n/2 
for all large n, to ensure that the sums from \an\ to n — \an\ are non-empty. If we define: 

[an] — 1 [an] — 1 

Sn = E X (^j)~ E X ( 2 >^)> 

n— [an] —1 

4 1 ) = e *(u)> 

j=[an] 
n— [an] — 1 

4 2 ) = e 

£= [an] 



we can rewrite LoserHasMoreThan(a, n) as 

LoserHasMoreThan(a,ra) = {-Eft < S n < E* 2 )}. 



(39) 



The random variable S n equals A r an -| , as defined in the proof of Theorem [H and the 

En^'s are akin to the B„s in that proof. Similarly to that proof, we note that En and 
(2) . 

E n are independent, identically distributed and independent from S n . Since S n has no 
point-masses in its distribution, 



Pr 



(LoserHasMoreThan(a, n)) 



1 _ Pr > Sn ) _ Pr ( Sn > 

1 _ Pr (_£ n > Sn ) _ Pr ( Sn > £ n ) 
Pr(|S n | < £7 n ), 



where £7 n = e4 . Now notice that: 



1. E n concentrates around its mean. Indeed 

E n = 



+00 

E 

j=[an] 



+00 

E 

j=n-[an] 
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Applying Corollary [T] to each bracketed term and noticing that (by Lemma [T]) 

Si(fan],n — fan]) = 0, (Si (fan])) , 
we conclude that there exists & D 1 > depending only on / and a such that 

Pr (\E n -Si(\an],n- fan])| > JLs^an]?) < D'e^ . 



2. The estimates on A n in (|36p imply that there exists a constant depending only on / 
such that for all e > and all n large enough 



Pr(|E n | < e) -c rcml e 



for the same sequence {c m } me N converging to c > appearing in the proof of Theo- 
rem m 

Putting those results together we conclude that 

Pr(|E n | < E n ) -c M [Si(fan],n- fan])] 

= O (^74 [Si ( f an] ) + Si (n- fan])] + [S x (fan]) - S 1 (n - fan] )] 3 + e~ nl/i ) 

o([Si(fan])) 

which implies that 

Pr( x y ) (LoserHasMoreThan(a, n)) = Pr (|A n | < C n ) ~ c [Si (fan] , n — fan])]. 
This is precisely the desired result. □ 



8 Extensions, related results and open problems 

• The proof of Theorem H] generalizes directly to the following statement. 

Theorem 6 Let q : N — ► N be a function and f be a valid feedback function such 
that 



Si 



n — on 



n + q(n) 



>n 7 S 2 



n — on 



n + q(n) 



as n — > +oo, for some constant 7 > depending only on f . Then for any fixed 
(x,y) £ N 2 there exists a constant c > depending only on f , x and y such that for 
n > 1 



Pr (XtV) (|/i(n- (x + y)) -/ 2 (n- (x + y))\ < q(n)) - cSi 



n — q(n) 



n + q(n) 
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Theorem U] is a special case of this result when q(n) = an, and the role of ([6]) is to 
show that Ylj^n^~qM) li ^"(*> -0 concentrates around its mean (cf. Lemma We 
omit the proof, but note the following corollary (take q(n) = Xy/n.) 

Corollary 4 Assume f(n) = n p for p > 1. Then the probability that the losing bin 
has at least (n — X\/n)/2 balls at the time when the total number of balls is n is 
asymptotic to 

{const.) x n 1/2 ~ p (n > 1), 
with a constant that depends on the initial conditions, p and X. 

We mention this corollary because it relates to a result of [B] , where it is shown that for 
a large initial number of balls n, the probability of monopoly by bin 1 has non-trivial 
behavior when |Ji(0) - J 2 (0)| = 9 (y/n). 

• We do not know any result more precise that Theorem [3] about the behavior of 
Pr[ t Q ] (HasMoreThan(/3, T)), but a related question has been addressed. Let f(x) = 
x p and recall that Moni is the event that bin 1 achieves monopoly. Assume we start 
with bin 1 losing, from state [t, a], < a < 1/2. We prove in [9] that 

Pr [t,a] (Morn) = exp(c p (a)t + o(t)) 

for some negative, smooth function &,(•) < 0. Moreover, we show that conditioned 
on Moni, the fraction of balls in the first bin approximately follows the solution to a 
deterministic ODE. 
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